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Approximations of the derivative f'(x) (5 i 0) 

1. + - f'(x) and '(* + <)-'(*-■») _> r (x) . 

o 2d 

2. Take g(y ) = ay + b such that 

J2jL-N V ( x + <5y) — ^ (-V H- «5y)| 2 is minimal. Then 


a = 


3 

2A/(A/ + Jj(A/+ 1)5 


N 


J2 f ( x+ 6 j)J ->• f'oo 

J=—N 


3. Take g(y) = ay + b such that \ f(x + 6t) - g(x + St )| 2 dt 
is minimal. Then 

a = ^ J i f(x + <$f)f<tf -> f'(x). 

A/ofe: For A/ = 1 item 2 reduces to item 1b. 

Note : For 5 in item 2 replaced bt 5/N we can see that a in 
item 2 tends to a in item 3 as N ->• oo. 





Item 3 as limit of item 2 for N -> oc 


For W-Hxiwe have 


1 

N 


N 

£ 

J=-N 


f X + 



9 



and 

3 N 2 1 A / #\ 

2^ (A/+l)(/V+1) N ,^ N \ X+ Nj 


f \f(x+6t)-g(x+5t )| 2 

s -> I /_’ f(x+w),d ' 





Graphs of f and g in the three cases 








Transfer function 

Suppose Dsf is an approximation of f'(x). The transfer function 
associated with D s is 


Hs{u) := e~ lujX D$(y ^ e luiy ). 


In the examples H s (uj) = iuh(du). Preferably, \h\ is close to 1 
for small argument and close to 0 for larger argument (low pass 
filter). 

For the previous three cases the transfer function can be 
computed to be: 



3. H$(uj) = iuj 


Hs(uj) = iu 


N + 1 








log-log plots of absolute values of transfer functions 
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For 


2A/(A/+1)(/V + 1) f 


E f (x+j)J (N = 1,2,3,4 in 


-N 


black, red, blue and orange, repectively. 

























Orthogonal polynomials 


/ Pm{x) Pn(x) d/j{x) = h n 5 m ,n, Pn{x) = k n X n + ■■■ 

Jr 

Let f be given. Minimize f R \ f(x + St) - g{t)\ 2 d/j,(t) where g is 
a polynomial of degree < n. Then g is the projection in Z_ 2 (/x) 
of f on the polynomials of degree < n : 

9( s ) = Yl^jr- I f ( x + st )Pj{t)dfj,(t). 

j=o h ) J K 

Then 5~ n g( n \ 0) ->• f^Xx) as 6 | 0. Indeed, 

s 'V"(0) = Tf / f(x + «)p„(f)°M0 

"n 0 Jr 

= TTXS I f(x + St)p n {t)dfj.{t). 

"n o Jr 

Now plug in the Taylor series for f(x + St)'. 





Taylor series 


Then 


n 

f(x + 5t) = fU\x) 
y=o 


M 

yi 


+ °((<^) n ) 


r" 9 <">(0) = ^ f 

nn o J 1 

k n n\ 


h n S n 


E 

y=o 


y! 


+ p n (f) d//(f) 


p n (f)f / Cf/i(f) + 0(1) 


= y {n) (x)^ / Pn (f)f n dM0 + o(i) 

r, n Jm. 

= y {n) (x) + o( i). 


At least, if p has bounded support. Otherwise assume that f 
has at most polynomial growth. 






Similarly with Peano derivatives 


Suppose that for certain c k 

n 

f (y) = ^2^(y- x ) k+ 0 (\y~ x \) n ) asy ^ x - 

k =0 

Then c n is called the n-th Peano derivative atx. This was 
introduced by Peano in 1891. 

Then the same approximation result holds, with h n \x) 
replaced by c n . 


Typical cases of orthogonal polynomials 

► // absolutely continuous: c///(x) = w(x) dx. 

For instance Jacobi polynomials p n (x) = Pn a,l3 \x) with 

a,/3 > - 1 and 

/*1 

J P%*’ p) (x) P^\x) (1 - x)“(1 - xf dx = 0 (n / m). 

In particular Legendre polynomials P n (x) = ptf'°\x ) 
(constant weight function on [-1,1]). 

► p discrete: 

P P™( x ) w ( x ) = 0 ( n / m ) 

xex 


with X a discrete subset of M. 

In particular, X may be finite: X = {x 0 , x-i ,..., x N }. Then 
only p n for n = 0,1,..., N. 


Special cases 

► f R g(t)d/i(t) = fh g(t)dt. 

Then p n (t) := P n {t) (Legendre polynomial). 

f(n> ^ = ^TnT'^ f\* + St)P M dt. 

► f R g(t)dfi{t) = a/- 1 E y =-w9 , (/7 /v )- 

Then p n (t) = Q n {-Nt + A/; 0,0. 2N) 

(special symmetric renormalized Hahn polynomials). 


f( n \x) 


= (2/7 + 1)! (2A/) n+1 1 

2 n+1 n! (2A/ + 1 ) n+1 < 54.0 5 n 

1 N 

x A/ E f ( x + 8j/N) Qn{N( 1 — j/N)\ 0,0,2/V). 

j=—N 


For n = 1 we have pi (t) = f in both cases. 







Connection with the continuous wavelet transform 

The continuous wavelet transform <t> g is defined by 

(V)(a. b ) : = l a l“ 1/2 / f ( f )(* - 6)) dt - 

Jr 

Here a,b6M,a/0, andt the wavelet g is a nonzero function 
in (Z. 1 n Z. 2 )(M) such that f R g(f) dt = 0. 

Now, fix n > 0, let c//x(Z) = w(t) dt with t t k w{t) in 
(Z. 1 n L 2 )(M) for k < n. Then we have a wavelet 

g(t) :=p n {t) W{t). 

Then 

[ f(x + t8 ) p n (0 w(f) dt = -T 1 / 2 *)■ 

ii 

-> /<">(*)■ 

h n d n ~2 


and 




History 

1. Cioranescu (1938) for c//j(x) = w(x) dx. Not picked up . 

2. Haslam-Jones (1953): Suppose the /7-th Peano derivative 
c n of f at x exists. Let J be a finite interval and let v be a 
signed measure on J such that fj t k dv(t) = 0 for k < n 
and / 0 for k = n. (For instance du{t) = p n {t) dp(t).) Then 


c n = lim 


n\ f,f{x + 5t)dv{t) 


< 54,0 5 n jjt n dv(t) 


Not picked up. 


3. Lanczos (1956): 
3 A/ 2 


N 


25 (A/+l)(/V + 1) N ^ N f { X+ li) N ^ f{x) 


and for N -> 00: — / f(x + 5t)tdt -)• ?(x). 

7-i 


A lot of follow-up: Lanczos derivative. 





4. Savitzky & Golay (1964) 


For low values of n they find the polynomial g of degree < n 
such that K( x + S J/ N ) ~ 9(j/ N )f is minimal. Then, for 

m < n, 5~ m g( m X 0) approximates f m \x). 

The authors were interested in spectroscopy. Their publication 
in the journal Analytical Chemistry had thousands of citations. 
The connection with orthogonal polynomials was later given by 
Meer & Weiss (1992). 

This work suggests a multi-term extension of our general 
approximation theorem of the n-th derivative. 


A multi-term variant 


Let g still be the orthogonal projection in L 2 (g) of t ^ f(x + 5t) 
on the space of polynomials of degree < n. So 

9( s ) = Y,^JT- I f ( x + st )Pj(t)dfj.(t)- 

j=o h i J K 

But now let m < n and consider 5~ m g (m) (0) as a possible 
approximation of f( m \x). Indeed one can show that 

S~ m g {m) {0) = ^ (l f ( x + 6t )Pj( f ) p) m \ 0) 

= f( m \x) + o{5 n ~ m ). 

For fixed m the approximation becomes better as n gets bigger. 
Write D™f := 5~ m g( m \ 0). 



Transfer function for the multi-term variant 


Moreover assume that // is an even measure and that n- m is 
odd. The transfer function is H$(oj) := e~ IUJX D™(y i->- e luiy ). 

Then, for some bounded function G: 

HsM = G(fo)). 

\Hs{u)\ stays close to u m for a while and then falls off. For fixed 
m the falling off starts at higher was n gets bigger. 

So bigger n means better approximation but more perturbation 
by noise. A best suitable n has to be found depending on the 
particular problem. 



Transfer function in Legendre case for m = 1 and 
n = 1,3 (log-log plot) 








The multi-term variant for m = 0: smoothing 

f(x) is aproximated by 

Y, Pj( ^ I f ( x +St)Pj{t)dfi{t)= I f{x + 5t)K n {t,0)d t i(t), 

y_0 n j JM. JR 

where 

„ ft ^ s^ P]{t)Pj{s) kn p n+: {t)Pn(s) - p n (t)p n ^(s) 

IS. n (,r, S) — > -r- — 7 ; -r-7--- 

hj k n+ 1 h n t — s 

is the Christoffel-Darboux kernel. 

Smoothing (earlier called graduation ) has old nineteenth 
century roots, most in the finite discrete case. It was for 
instance important in actuarial sciences. We will change 
notation. 






Smoothing in the finite discrete case 

Let p be a function defined on {-A/, -A/ + 1,..., A/} such that 
p(-x) = p(x). We go from an input function f to an output 
function g by convolution with p\ 

N 

g(y)= f (y~ x )p( x )- 

x=—N 

In particular, take for some n < N 


p{x) = K 2 n (x, 0) w(x) = P2jXp2 ^°^ W (x ), 

j =o 2; 

where the are orthogonal polynomials on 

{-A/, -A/ + 1,..., A/} with respect to even weights w(x). 

Then the smoothing reproduces the polynomials of degree 
< 2n + 1, but not all of degree 2 n + 2. Such smoothing is called 
exact for degree 2n + 1. 



The characteristic function 


For smoothing by convolution with p the characteristic function 
is given by 

N 

#*):= E Pi x ) e ~'“- 

x=-N 

The smoothing is exact for degree 2 n + 1 iff 

4>(uj) = 1 - au 2n+2 h -for some a / 0. 

Define p m by 

mN N 

f (y~ x ) Pm{x) ■■= f{y-X:-...-x m )p{xi)...p{x m ). 

x=~mN Xi,...,x m =-N 

De Forest (1878): When can asymptotic behaviour of p m {x) for 
m oo be described? 

Schoenberg (1948): Iff |0(w)| < 1 for 0 < to < 2ir. 

Then the smoothing is called stable. Then a above is positive. 


Greville’s work 


Greville (1966) established stability for p(x) corresponding to 
the weights of certain special symmetrized Hahn polynomials 


w Q {x) 


/a + N + x\ /ct + A/ — x\ 

V N + x ) V N-x ) 

(x = ~N,-N + 1,..., N - 1, N) 


for a = 0,1,2,... . 

He also took the limit for a —> oo. Then 





the weights of special symmetrized Krawtchouk polynomials. 
However, he did not explicitly use orthogonal polynomials. 


The symmetric Krawtchouk case 

On the one hand 

N 

(j)(uj) = ^2 e ~ IXUJ K 2 n(^, 0 ) w(x) = 1 - auj 2n+2 H- 

x=-N 

= 1 - (sin 2 o;/2) n+1 P(cos 2 w/2) 

for some polynomial P of degree N - n - 1 . 

On the other hand 

N 

<fi k \ tt)= (-/x) ,t (-1) Jf K 2n (x,0)w(x) = 0 if Ac < 2N - 2n 

x=-N 

(use that P 2 n{*) = (-1 ) N+X )■ Hence 

= (cos 2 u/2) N ~ n Q(s\n 2 u/2) 
for some polynomial Q of degree n. 


The symmetric Krawtchouk case (continued) 

Thus P of degree N - n- 1 and Q of degree n are related by 

1 = z n+1 P( 1 - z) + (1 - z) w ~ n Q(z). 

Q(z) = power series of (1 - z)~ N+n in z truncated after term of z n 

= ^2 ^ zk i so Q(sin 2 w/2) > 0 so 4>(uj) >0 (0 < u < ir). 

k=0 


P(1 - Z) 

= power series of z~ n_1 in 1 - z truncated after term of (1 - z) A/_n_1 

N-n -1 


E 

k=0 


Hence 
1 = z 


N-n -1 , .. 

_ - 7 n+^ ( n + 1 )k 


E 

k =0 


k\ 


(1 -z)* + (1 -z) 


A/—n 


E 

/<=o 


(N ~ n )k k 
k\ 


(identity with two incomplete binomial series) 






The identity with two incomplete binomial series 


k=0 ^ ' k =0 ^ 


History, variations and extensions considered in Schlosser & K 
(2008), triggered by the occurrence of the identity in an 
unpublished manuscript (2007) by P. de Jong. 

We then traced back the identity to Chaundy & Bullard (1960). 
Their proof: 

m —1 

i=?"E 

k =0 


m + n - 1 




(1 -z)‘ 


n -1 


+ (i-z)”£ 


£=0 


m + n - 1 


— Z^Pm,n(1 


^Z^(1 - Z) n 1 1 

-z) + (1 -z) m P n , m (z). 


Now reason as before (Q follows already by symmetry from P). 


Earlier occurrences of the identity: Hering (1868) 


A. G. Hering, Summation der n ersten Glieder der binomischen 
Reihe mittelst der Theorie der hypergeometrischen Reihen, 
Programm der Realschule in Chemnitz, 1868; JFM 01.0089.04. 

Durch Substitution dieser Werthe in die obige Gleichung erhalten 
wir ans 3) des vorigen §: 

(i _ x )7“ = (i - *)— - (i - *)-* f 1 - r -*£"• h 


Earlier occurrences of the identity: de Moivre (1738) 

A. de Moivre, The doctrine of chances, London, second ed., 
1738 . 

But as there is a particular elegancy for the Sums of a finite num¬ 
ber of Terms in thole Scries whole Coefficients are figurate num¬ 
bers beginning at Unity, I IhalHef down the Carton for thofe Sums. 

Let n denote the number of Terms whofe Sum is to be found,, 
and p the rank or order which thofe figujate numbers obtain, then, 
the Sum will be 


* * i » 

1 x nx _ 



’ Uc - 


which is to be continued till the {mpiber of Terms be =/. 

The series with n terms of figurate numbers of order p is: 








Earlier occurrences of the identity: de Montmart 
(1713) 

Pierre Renard de Montmort, Essay d’analyse sur les jeux de 
hasard, 2nd edition, 1713 


AuTM FoRMUtt. 

19 1. L’a n a l y s e m’a encore fourni une autre formuJe. 
Suppofant les mfimes denominations que ci-deflus, je 


-1 


.b?~* 


_ , „. 1 x<*r** ?x«t>' r-p’ 

irouv.de fort de Pierre + 77: 

Etd = 

mSmc le fort de Paul=+ 7^=^T 


f. f ->■ t.f -Kit* 1 ** 

i. 1.4 


v • f 1 • 




; 1»4 


. x o-t-i 1 

&C. 











idem, translated 

Pierre Renard de Montmort, Essay d’analyse sur les jeux de 
hasard, 2nd edition, 1713, translated. 


If we suppose that the number of chances that Pierre has to win each point, or if we 
wish that his strength be to that of Paul as a to 6; we will have likewise the lot of Pierre 
by multiplying the terms of this series which are the coefficients of the power in, by the 
powers of a & of b which correspond to them (art. 27): thus the preceding series becomes 
a m b°+ma ,n ~ 1 b+ 

which it is necessary to continue to the number of terms expressed by q, & to divide 
by (a + b) m . The formula which designates the lot of Paul is 1 x b m a° + mb 7n ~ 1 a 
2 qq _|_ nt.m-j.m-2 ftm-3 Q 3 _j_ m.m-1 ,m-2.m-3 tfn- 4 Q 4continued tO 

the number of terms expressed by p, & divided by (a + b) m . 

ANOTHER FORMULA. 


191. The Analysis has again furnished me another formula. Supposing the same de- 
nominations as above, I find the lot of Pierre = |*“^° + + f.'j’.fa+t ?)r+ 3 + 

p-p+l-p+Sxan 3 I p.p-l-l.y+2.p+3xa |, 6 1 „ , , jjk : of Paul = lxtlla ° + t xb '' al + 

l,2.3.(a+f.)r+ 3 + 1.2.3.4.(o+6)>>+‘ + AnU UKeW1Se 01 rdul (a+b)i + (a+6)«+ l + 

q.q+1 Xb q a 2 , q.q+l .q+2 X b q a 3 , q.q+l.q+2.q+3xb q a‘ l . p 

1 . 2 .( a + 6)«+ 2 ^ 1 . 2 . 3 .( o + 6)«+ 3 " r 1 . 2 . 3 . 4 .( a + f>) 9+4 ' 

The formula which expresses the lot of Pierre will have as many terms as there are units 
in q, & that which expresses the lot of Paul as many terms as there are units in p. 



















The problem of points 


This comes from a game of chance with two players Pierre and 
Paul who have chances p and 1 - p, respectively, of winning 
each round. The player who has first won a certain number of 
rounds will collect the entire prize. Suppose that the game is 
prematurely interrupted when Pierre has to win still n rounds 
and Paul m rounds. What is then a fair division of the stake? 
First solution: Suppose m + n - 1 rounds are still played. 
Pierre: Ek=o ( m +^)p m + n ~ k -\- I - p) k 

Paul: o (TVC 1 -p) m + n -i-' 

Second solution: Suppose they still play until there is a winner 
(at most m + n - 1 rounds). 

Pierre: P ES (Tl^'d -p)* 

Paul: (1 - p) Etl r^"') P S 0 - P)”- 1 
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